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EXECUTIVE  SUMMARY 

The  role  of  a decision  maker  is  a difficult  job  and 
in  the  case  of  a Program  Manager  (PM)  his  task  of  decision 
making  may  be  even  more  difficult  because  of  the  complexity 
and  interdependency  of  factors  related  to  typical  problems 
he  must  resolve.  This  study  examines  some  decision  analysis 
techniques  that  are  applicable  to  program  management  and 
recommends  an  approach  for  use  by  a PM  to  aid  in  his  decision 
making. 

The  approach  recommended  is  a modified  version  of  a 
process  that  the  Army  calls  estimate  of  the  situation. 

Using  the  estimate  of  the  situation  as  a guide,  a somewhat 
different  format  for  the  process  is  recommended.  The  new 
format  is  called  Estimate  of  zhe  Circumstances.  Other 
procedures,  such  as  requiring  each  staff  section  chief  to 
prepare  an  estimate  for  his  point  of  view,  remained  the  same. 
Content  of  the  estimate  was  changed  to  reflect  information 
required  and  applicable  for  nrogram  management. 

A system  for  collecting,  analyzing  and  evaluating  in- 
formation required  as  input  to  the  estimation  report  is  also 
proposed.  The  system  contains  a Program  Management  Infor- 
mation System  (PMIS),  a network-time  oriented  system  and 
analytical  techniques  which  in  some  cases  may  provide  adequate 
information  for  decision  making.  However,  for  complex  problems 
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all  elements  of  the  system  are  used  to  produce  an  adequate 
analysis  and  evaluation. 

A comparison  of  this  system  is  made  with  the  system  used 
to  design  communications  systems  such  as  radios,  radars  and 
computers.  Results  of  the  comparison  show  that  the  systems 
are  identical  and  that  in  view  of  accuracies  achieved,  par- 
ticularly with  computers,  similar  accuracies  in  estimating 
and  predicting  results  of  problems  faced  by  a PM  could  be 
achieved.  Although  these  procedures  can  produce  excellent 
results,  it  is  noted  that  they  do  not  in  themselves  provide 
the  solution  but  a solution.  The  PM  must  still  decide  if  the 
solution  derived  through  the  use  of  these  techniques  is 
acceptable. 

It  is  concluded  that  in  spite  of  the  shortcomings  of 
the  proposed  system. excellent  approximations  can  be  achieved 
and  that  the  estimate  of  the  circumstances  procedure  does 
provide  the  PM  with  a capability  for  making  sound  and  logical 
decisions . 
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DECISION  ANALYSIS 


FOR  THE  PROGRAM  MANAGER 


Introduction 

During  the  life  cycle  of  major  weapons  system  acquisition 
a Program  Manager  (PM)  must  make  numerous  decisions  that  often 
involve  selecting  an  optimum  or  best  course  of  action  for  accom- 
plishing various  tasks.  Many  of  these  decisions  involve  com- 
plex and  interrelated  factors  that  require  detailed  analysis 
and  evaluation.  Complicated  problems  of  this  nature  demand 
a comprehensive  and  thorough  decision  process  if  sound  and 
logical  decisions  are  to  be  made.  Although  there  are  many 
decision  analysis  procedures  or  systems  none  of  them  are  com- 
pletely capable  of  incorporating  the  many  varied  and  intangible 
factors  a PM  must  consider.  This  report  examines  some  of  the 
methods  that  can  be  utilized  and  proposes  a system  that  could 
assist  the  PM  in  making  decisions. 

The  proposed  system  involves  the  use  of  a Program  Manage- 
ment Information  System  (PMIS),  a network  system,  analytical 
techniques  and  what  is  called  decision  tree  analysis.  It  is 
demonstrated  how  some  of  these  techniques  in  themselves  provide 
adequate  information  for  decision  making  for  those  problems 
that  are  relatively  simple.  For  the  complicated  problems  it 
is  shown  how  all  of  these  techniques  are  used  in  conjunction 
with  each  other  to  provide  information  that  is  combined  and 
evaluated  using  a proposed  format  called  an  estimate  of  cir- 
cumstances . 
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In  chapter  two  decision  analysis  techniques  are  discussed 
in  general  to  include  the  strong  points  and  pitfalls  that  one 
must  recognize  when  using  these  techniques.  Also  discussed 
is  the  business  management  orientation  of  current  decision 
analysis  techniques  and  the  lack  of  any  significant  amount  of 
literature  on  applications  for  project  management.  Some  of  the 
methods  applicable  to  project  management  is  examined  in  chapter 
three.  An  explanation  of  each  method  is  given  along  with  how 
they  are  to  be  used  in  the  system  proposed. 

Chapter  four  discusses  analytical  techniques  applicable 
for  analyzing  and  evaluating  such  critical  factors  as  cost, 
schedule  and  technical  performance.  It  is  shown  how  to  esti- 
mate, evaluate,  and  analyze  these  factors  and  determine  criti- 
cality of  their  relationship  to  the  problem  to  be  resolved. 
Chapter  five  discusses  psychological,  social,  cultural  and 
organizational  factors  that  must  be  considered  when  r.a  ing  a 
decision.  Since  these  factors  are  not  easily  evaluated  using 
analytical  methods,  a subjective  means  for  handling  these 
factors  is  proposed. 

The  estimate  of  circumstances  format  is  discussed  in  chapter 
six.  It  is  shown  how  through  the  use  of  this  format  all  data 
provided  from  analysis  techniques  previously  described  is 
combined,  analyzed,  and  evaluated.  It  is  this  evaluation 
process  that  provides  the  PM  with  adequate  information  to  make 
a decision. 
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Some  typical  critical  problems  or  decisions  a PM  must  make 
during  all  phases  of  life  cycle  development  are  examined  in 
chapter  seven  and  it  is  shown  how  previously  discussed  analysis 
techniques  can  be  used  to  assist  the  PM. 

Chapter  eight  contains  the  summary  and  conclusions  and  it 
is  concluded  that  decision  analysis  techniques  are  effective 
means  for  assisting  the  PM  in  making  decisions. 
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DECISION  ANALYSIS  PROCESSES 

Decision  analysis  techniques  are  defined  as  any  process 
that  provides  a logical  and  orderly  means  for  defining  a 
problem;  determining  all  critical  factors  that  have  impact 
on  the  problem;  selecting  feasible  solutions;  analyzing  and 
evaluating  alternatives  and  providing  a possible  solution.  It 
is  appropriate  to  emphasize  that  decision  analysis  techniques 
do  not  provide  the  solution  but  a solution  to  a problem.  The 
decision  maker  still  must  decide  if  the  solution  derived  from 
the  analysis  is  optimum  for  resolving  his  problem.  In  other 
words,  the  results  of  an  analysis  technique  is  accurate  to  the 
extent  of  the  accuracy  of  quanitative  data,  facts  and  assump- 
tions used  in  the  analysis.  Even  if  all  data  and  etc.,  used 
is  accurate,  validity  of  the  results  could  be  questioned  be- 
cause results  are  only  approximations  of  what  could  or  may 
happen  given  certain  circumstances . However,  regardless  of 
the  approximate  nature  of  results,  decision  analysis,  if 
conducted  properly,  provides  a decision  maker  with  considerable 
knowledge  and  insight  as  to  inter-relationships  of  critical 
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factors  pertaining  to  a problem  and  with  such  knowledge  the 
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making  of  a sound  and  logical  decision  is  greatly  enhanced. 

Most  of  todays  literature  on  decision  analysis  is  oriented 
toward  business  management  or  the  engineering  sciences  where 
problems  are  generally  structured.  Such  structured  problems 
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are  readily  adaptable  to  decision  analysis  techniques.  Con- 
sequently, very  good  approximations  are  obtained.  The  PM's 
problems  are  generally  unstructured  and  although  some  of  the 
literature  address  unstructured  problems  and,  in  some  cases, 
discuss  the  similar  problems  of  PMs,  there  isn't  enough  data 
available  that  a PM  could  review  to  learn  analysis  techniques. 

The  unstructured  nature  of  a PM's  problems  stem  from  many 
intangible  and  difficult,  if  not  unpredictable^  factors  such  as 
life  cycle  cost,  resolution  of  a technical  problem  before  it 
delays  entire  projects  and  will  a modification  or  engineering 
change  proposal  (ECP)  provide  the  required  performance.  These 
factors  are  difficult  to  quantify  but  must  be,  if  analysis 
techniques  are  to  be  applied  for  resolution  of  typical  problems 
faced  by  a FM. 

Most  analysis  techniques,  as  mentioned  previously,  are 
designed  for  structured  problems  and  the  few  that  are  adaptable 
to  unstructured  problems  may  require  some  minor  adjustments 
before  they  can  be  applied  to  most  problems  of  a PM.  In  the 
next  chapter  some  techniques  that  are  adaptable  will  be  dis- 
cussed and  it  will  be  shown  that  although  the  techniques  may  not 
be  perfect  for  a FM. 's  proble-s,  with  due  considerations  for  the 
unstructured  nature  of  the  problem,  an  adequate  approximation 
can  be  obtained. 
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MANAGEMENT  TECHNIQUES 


Project  management  is  a difficult  task,  and  traditional 
organizational  structure  and  management  techniques  have  proved 
awkward  or  ineffectual  in  accomplishing  it.  The  PM  is  respon- 
sible for  getting  the  job  done  on  schedule  within  allowable 
cost.  However,  he  must  often  accomplish  his  goal  by  employing 
the  efforts  of  many  separate  organizations  and  individuals, 
most  of  which  are  not  under  his  direct  control.  The  techniques 
discussed  in  this  chapter  provide  the  basic  tools  needed  by  a 
PM  in  planning  , scheduling  and  controlling  the  work  under 
these  conditions. 

The  PM's  primary  task  is  to  direct  and  coordinate  toward 
one  goal  the  work  of  several  groups  involved  in  the  project; 
yet,  the  complexity  of  todays  operations  forces  him  to  divorce 
himself  from  details,  to  deal  only  with  the  broad  aspects  of 
the  problem.  He  is  inclined  to  think  and  act  in  generalities 
because  he  lacks  the  techniques  or  management  aids  which  would 
enable  him  to  comprehend  the  whole  operation  in  detail.  He 
does  not  always  know  which  activities  are  critical  and  require 
special  attention,  or  what  effect  a delay  or  failure  in  one 
activity  will  have  on  others  following  it  or  on  the  success  of 
the  project  as  a whole.  What  is  needed  is  a system  which  will 
provide  the  PM  with  an  up-to-date  picture  of  the  operation  at 
all  times,  and  which  would  follow  a uniform  system  understood 
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by  all.  The  methods  of  this  chapter  are  designed  to  fill  these 
needs . 

The  first  arid  probably  the  most  impoprtant  component  of  the 
proposed  decision  analysis  system  is  the  Project  Management 
Information  System  (PMIS).  A PMIS  in  its  broadest  sense  is 
defined  as  a set  of  policies,  models,  procedures  and  files  of 
information  which  operate  to  record,  manipulate,  store,  retrieve, 
process  and  display  information  useful  in  planning,  scheduling, 
executing  and  controlling  complex  projects,  which  cut  across 
organizational  and  functional  lines  and  which  must  achieve  the 
specified  results  at  a particular  point  in  time  and  within  a 
given  cost  budget.  (I)1 

A PM's  most  critical  problems  all  relate  to  cost,  schedule, 
and  technical  performance  in  some  manner.  Therefore,  the  pri- 
mary objectives  of  a PMIS  should  be  to  obtain,  analyze  and 
evaluate  data  of  this  nature.  To  develop  such  a system  the 
Work  Breakdown  Structure  (WBS)  should  be  used  as  the  basis  for 
establishing  a reporting  system  under  the  Cost/Schedule  Con- 
trol Systems  Criteria  (C/SCSC)  established  by  the  Department 
of  Defense  (DOD). 

With  C/SCSC  the  PM  has  an  effective  means  for  keeping 
track  of  overall  contractor  progress.  The  basic  unit  for 
tracking  contractor  performance  is  the  monthly  Cost  Performance 


This  notation  will  be  used  throumnout  this  Study  Report  for 
sources  of  quotations  and  major  references.  The  number  in  paren- 
thesis is  the  source  listed  in  the  Bibliography. 
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Report  ( CPR ) . Prom  the  CPR  the  PM  is  provided  data  on  work 
package  completion  and  scheduled;  funds  expended  and  projected 
expenditure;  material  committment  and  expenditure  and  variance 
analysis  of  cost  and  schedule.  With  this  data  a Performance 
Analysis  Curve  can  be  constructed  using  (1)  Budgeted  Costs 
for  Work  Scheduled  (BCWS);  (2)  Budgeted  Cost  for  Work  Per- 
formed (BCWP);  and  (3)  Actual  Costs  for  Work  Performed  (ACWP). 
With  these  three  factors  then  cost  and  schedule  variances  can 
be  established.  The  cost  variance  is  the  difference  between 
the  BCWP  and  the  ACWP.  At  any  point  in  time  it  shows  whether 
the  work  actually  performed  has  cost  more  or  less  than  budgeted. 
The  schedule  variance  is  the  difference  between  BCWS  and  BCWP. 

At  any  point  in  time  it  shows,  in  dollars,  the  difference  be- 
tween work  scheduled  to  be  accomplished  versus  work  that  was 
actually  accomplished. 

Figure  1 shows  two  examples  of  the  Performance  Analysis 
Curve.  Figure  la  shows  that  the  program  is  on  cost  and  schedule. 
Although  this  situation  could  happen,  it  is  somewhat  unrealistic 
and  should  be  investigated.  Figure  lb  shows  that  the  program 
is  behind  schedule  and  over-run  in  cost.  This  situation  indi- 
cates technical  difficulties. 

C/SCSC  provides  the  PM  adequate  information  on  contractor 
performance  but  nothing  about  other  factors  such  as  various 
supporting  agencies,  commands  and  environmental  factors  that 
may  affect  the  program.  When  dealing  with  supporting  agencies 
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C/SCSC  types  of  reports  should  -be  used  wherever  applicable 
and  practical.  It  is  recognized  that  supporting  agencies 
in  all  services  are  not  organized  and  validated  under  C/SCSC. 
However,  the  Army  has  started  to  organize  some  of  its  arsenals 
under  this  concept  and  it  is  anticipated  that  in  the  near 
future  all  services  supporting  activities  will  be  validated 
under  C/SCSC.  Until  this  is  accomplished  the  PM  should, 
through  memorandums  of  agreements,  negotiate  between  himself 
and  the  supporting  command  reporting  procedures  similar  to 
the  CPR . 

For  environmental  factors,  a system  of  liaison,  periodic 
visits  and  simply  keeping  attuned  to  what  is  happening  outside 
of  the  PM's  organization  should  be  established.  This  area  will 
be  discussed  in  Chapter  five. 

The  PKIS  envisioned  gives  the  PM  a means  for  relating  cost, 
schedule  and  progress  toward  developing  a weapons  system. 

Using  this  system,  particular  areas  of  difficulty  can  be  iden- 
tified; but  hov;  the  problem  relates  to  the  overall  project 
cannot  be  ascertained.  To  provide  this  visibility  a network 
system  based  on  time  is  essential.  A network  time  oriented 
system  is  defined  as  an  operations-related  system  that  tells 
the  manager  when  and  now  (in  terms  of  sequence  of  operations) 
the  program  will  be  accomplished.  (1) 

The  development  of  a network  system  consists  of  three 
phases.  The  phases  are  (1)  planning;  (2)  scheduling;  and  (3) 
monitor  and  control.  The  planning  phase  consist  of  breaking 
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down  the  project  into  discrete  activities  and  events  and 
arranging  these  in  logical  sequence  in  the  network. 

The  schedule  phase  consists  of  estimating  the  duration 
of  each  activity  and  then  totaling  these  times,  proceeding 
logically  through  the  network,  to  determine  overall  duration 
of  the  project.  If  this  procedure  is  reversed,  starting  at 
the  end  event  and  subtracting  the  activity  durations,  you 
can  derive  useful  schedule  information  including  identifying 
the  critical  path,  the  longest  path  of  activities,  through 
the  network.  Its  length  determines  the  project  duration,  and 
all  activities  not  on  this  path  may  have  delay  time  called 
slack. 

In  the  monitor  and  control  phase,  as  actual  completion  of 
each  activity  is  reported,  changes  are  made  and  a new  analysis 
determines  impact  of  progress  and  changes  on  the  future  plan 
and  schedule.  The  network  plan  must  be  periodically  updated 
and  analyzed  if  it  is  to  be  a viable  management  measure.  (1) 

Figure  2 shows  a simple  network  system  (plan),  bote  that 
activity  A must  be  completed  before  activities  B or  C can  be 
started.  Also  note  that  activities  R,  S,  and  T must  be 
completed  before  activity  W can  begin.  The  critical  path  is 
shown  by  the  double  line.  I.' umbers  between  events  represent 
estimated  times  required  to  complete  an  activity  before 
proceeding  to  the  next  phase. 

The  WBS  provides  an  excellent  means  for  breaking  the  project 
down  into  discrete  elements  and  should  be  used  in  conjunction 
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with  the  Advanced  Development  and  Program  Management  Plans 
to  develop  the  network  plan.  Successful  integration  of  all 
activities  of  these  plans  into  the  Network  system  provides 
the  PM  a valuable  tool  for  determining  the  impact  of  a par- 
ticular action  on  the  overall  program. 

Once  all  pertinent  information  and  the  inter-relationships 
have  been  determined  using  the  PMIS  and  the  Network  System 
some  problems  can  be  evaluated  with  analytical  techniques. 

The  basic  techniques  required  in  the  proposed  decision  anal- 
ysis system  are  discussed  in  a general  manner  in  this  chapter. 

A more  detailed  explanation  is  covered  in  chapter  four. 

Three  techniques  are  used.  They  are  sampling,  estimation, 
and  prediction.  All  three  involve  the  use  of  subjective 
judgments  to  some  extent.  For  example,  a PM  is  constantly 
faced  with  problems  of  estimating  costs,  schedules  and  technical 
performance.  The  techniques  used  in  estimating  will  not  elimi- 
nate the  uncertainity  but  will  give  the  PM  enough  data  which 
indicates  whether  his  estimate  is  reasonable.  Using  this 
estimate  the  PM  can  then  predict  if  certain  actions  can  be 
completed  on  schedule  or  whether  a certain  technical  parameter 
can  be  obtained.  As  mentioned  earlier,  these  methods  cannot 
and  should  not  be  expected  to  provide  the  solution  but  a 
reasonable  approximation  of  what  may  happen  under  given  cir- 
cumstances is  obtained. 

These  techniques  are  used  for  the  relatively  simple  problems 
that  do  not  involve  complex  relationships.  For  these  types  of 
problems  decision  tree  analysis  should  be  applied. 
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A decision  tree  is  a network  of  branches  corresponding  to 
possible  alternatives  with  subsequent  sequences  of  acts  and 
events,  stemming  from  an  origin  at  the  left  to  a time  horizon 
at  the  right.  Acts  (represented  by  squares)  are  choices  that 
one  controls  or  can  make.  Events  (represented  by  circles) 
are  possible  outcomes  determined  by  chance.  A path  through 
the  tree  represents  a possible  sequence  of  acts  and  events 
characterized  by  a value  at  the  right  of  the  diagram.  This 
value  may  represent  a loss  or  a gain.  (3) 

Figure  3 shows  a completed  decision  tree  for  a hypotnetical 
problem.  As  PM,  you  must  decide  whether  or  not  to  accept  a 
controversial  and  critical  engineering  change  proposal  (ECP) 
submitted  by  a contractor  during  the  latter  stages  of  full 
scale  development.  After  analyzing  the  problem  you  know 
that  without  the  ECP  life  cycle  cost  (LCC)  may  increase  (loss 
of)  by  $60,000,000.  Also  you  know  that  there  is  a fifty- 
fifty  chance  that  the  ECP  will  be  successful.  If  you  decide 
to  reject  the  ECP,  the  program  is  cancelled  and  a loss  of 
860,000,000  is  incurred.  If  you  accept  the  ECP  and  if  it  works 
your  loss  is  then  320,000,000.  You  are  then  faced  with  the 
problem  of  deciding  whether  to  stop  production  and  redesign 
for  a total  loss  of  0140,000,000  or  to  continue  and  try  to 
retrofit  models  at  a later  date  with  the  ECP  for  a loss  of 
$10,000,000.  Evaluating  this  tree  diagram  using  expected 
value  and  rollback  techniques  show  that  the  decision  has  to 
be  made  between  accepting  the  ECP  with  an  expected  loss  of 
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$45,000,000  or  rejecting  the  ECP  with  a known  loss  of 
$60,000,000.  A detailed  explanation  of  tree  diagram  con- 
struction, expected  values  and  roll  back  techniques  is 
contained  in  references  3 and  4.  The  intent  here  is  to  mere- 
ly show,  without  detailed  proofs,  that  a tree  diagram  can 
provide  the  PH  a valuable  tool  for  analyzing  complex  prob- 
lems . 

The  probabilities  used  in  the  diagram  can  be  known  or 
estimated.  A means  for  estimating  these  probabilities  and 
establishing  values  (dollars)  for  use  in  the  diagram  will  be 
covered  in  the  next  chapter. 


SAMPLING,  ESTIMATION  AND  PREDICTION 


Analytical  techniques  of  sampling,  estimation  and.  pre- 
diction of  outcomes  are  powerful  techniques  for  coping  with 
the  many  uncertainties  associated  with  project  management. 
Properly  applied  and  evaluated  these  methods  will  considerably 
reduce  the  degree  of  uncertainty  of  many  problems. 

By  reducing  uncertainty,  it  is  meant  that  these  methods 
will  provide  a PM  with  a means  of  determining  how  probable  a 
particular  event  will  occur  or  the  probability  of  success 
of  a particular  course  of  action. 

The  mean  and  standard  deviation  are  the  two  most  important 
values  from  which  inferences  about  a process  can  be  made.  In 
subsequent  paragraphs  derivations  of  these  values  using 
sampling  and  estimation  will  be  examined  and  shown  how  they 
can  be  used  to  predict  outcomes. 

The  sampling  technique  involves  observing  an  experiment 
and  recording  the  outcome  values.  Assuming  that  the  experi- 
ment is  conducted  a number  of  times  and  all  outcome  values 

are  recorded  a sample  mean  can  be  determined.  The  sample 

n 

mean  is  defined  as  X •-  >_  x,  . In  other  words  record  the  out- 
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come  values  and  determine  the  numerical  average.  From  this 

average  the  sample  variance  can  be  determined.  The  sample 

variance  is  defined  as  S “ 2_  (x*-x)  /(n-1).  The  standard 
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deviation  is  the  square  root  of  S • 
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With  these  values  one  can,  through  the  use  of  standard 
formulas,  estimate  and  predict  the  probability  of  occurrance  of 
particular  events.  (By  standard  formulas  reference  is  made 
to  tee  binomial,  negative  binomial,  geometric  and  hypergeo- 
metric distribution  formulas.  For  a detailed  explanation  of 
these  procedures  refer  to  reference  3«)  ln  addition  to  the 
standard  formulas,  the  concept  of  conditional  probability  and 
the  Bayesian  method  of  estimation  can  be  used.  (See  reference  4). 

If  large  samples  are  taken,  the  mathematical  laws  of  prob- 
ability guarantees  that  the  sample  means  approaches  the  true 
mean  with  certainty.  In  particular,  the  law  of  large  numbers 
states  that  as  the  sample  size  increases,  the  difference  between 
the  true  mean  and  the  sample  mean  becomes  negligible.  However, 
the  lav:s  also  show  that  the  sample  mean  is  an  unbiased  estimate 
of  the  true  mean  and, . therefore , the  sample  variance  is  a good 
estimate  of  the  true  variance.  (4) 

In  addition  to  determining  the  means  and  variances  of 
estimates  one  can  determine,  again  from  standard  formulas,  the 
probability  of  error  in  these  estimates.  Therefore,  knowing 
that  an  error  of  5 percent  or  10  percent  may  exist  in  an  estimated 
outcome,  allowances  or  compensations  could  be  made  to  accommodate 
the  deviations. 

The  last  technique  considered  is  that  of  hypothesis  testing. 
This  technique,  using  the  Neyman-Pearson  Criteria,  is  a most 
powerful  test  for  estimating  whether  a simple  hypothesis  is 


true.  (5)  This  method  is  far  superior  to  the  Bayes  method  in 
that  knowing  the  a priori  probability  and  cost  data  are  not 
necessary.  All  that  is  needed  is  the  distribution  function  of 
the  two  hypotheses  and  a determination  of  an  appropriate 
significance  level.  This  method  assures  that  the  optium  trade 
off  between  making  a Type  I or  Type  II  error  is  accomplished.  (5) 
The  procedures  described  thus  far  can  be  used  to  some  extent 
individually  or  in  conjunction  with  each  other  to  evaluate  many 
problems  faced  by  a PM.  For  example,  is  the  mean  flying  time 
of  a missle  equal  to  a certain  value;  should  a lot  of  ammunition 
be  accepted  if  a certain  number  of  defective  rounds  are  found  by 
inspection;  is  the  probability  of  a hit  by  a missile  equal  to 
.8  and  many  similar  problems  can  be  evaluated. 

These  techniques,  if  used  properly,  provide  good  approxi- 
mations for  a PM's  problems.  In  the  engineering  sciences  and 
particularly  the  electronics  area,  systems  are  designed  using 
these  techniques.  For  instance,  communication  systems,  to  in- 
clude radios,  radars  and  computers  are  designed  and  constructed 
through  the  use  of  methods  outlined  in  this  chapter.  Experience 

has  shown  that  these  systems  and  especially  computers,  have  an 
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error  rate  of  approximately  10  . (6) 

These  communication  systems  all  follow  the  same  decision 
process.  First,  the  incoming  signal  is  sampled;  second,  based 
on  the  average  value  of  the  sample,  which  is  the  same  as  the 
sample  mean,  a decision  is  made  as  to  what  signal  was  sent. 

In  the  case  of  computers,  the  decision  is  between  two  equally 
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likely  siganls  called  zero  or  one.  When  deciding  between 
two  equally  likely  situations  the  probable  error  is  .50  per- 
cent but  the  computer  makes  millions  of  decisions  per  minute 
on  signals  like  these  and  yet  makes  approximately  1 error  in 
about  10,000  and  sometimes,  100,000  decisions.  Thus  these 
analytical  techniques  are  powerful  tools  and  can  be  used  to 
assist  a PM  in  deciding  between  alternatives. 
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ENVIRONMENTAL  FACTORS 


Environmental  factors  are  intangibles  that  must  be  con- 
sidered when  confronted  with  making  a decision.  In  this  paper 
these  intangibles  are  considered  to  be  political,  social,  and 
cultural  matters  that  arise.  These  factors  occur  internally 
in  the  PM's  office  and  within  supporting  operating  commands 
and  higher  headquarters.  Recognizing  and  accommodating  all  of 
these  factors  becomes  a difficult  if  not  impossible  task. 

Trade  offs  or  balancing  these  factors  to  achieve  the  optium/best 
solution  is  the  most  one  could  hope  to  accomplish.  This  chapter 
examines  a simple  technique  that  could  be  used  for  balancings. 

To  insure  that  the  PM's  office  has  accurate,  and  timely 
information  regarding  these  factors,  an  effective  liason  system 
between  all  organizations  to  include  the  contractors,  must  be 
established  and  maintained.  Each  staff  section  of  the  PM's 
office  must  interface  with  their  counterparts  of  organizations 
involved  and  the  PM  must  establish  the  contact  between  the 
heads  o.f  the  organizations  concerned.  If  pursued  diligently 
this  approach  enhances  receipt  of  information. 

Once  all  factors  have  been  identified  a determination  of 
their  relative  importance  and  effect  upon  a particular  course 
of  action  must  be  made.  To  determine  the  impact  is  rather  easy 
but  to  establish  relative  importance  is  difficult  if  only  sub- 
jective means  are  used.  Therefore,  combining  subjective  evalu- 
ations and  techniques  described  in  the  last  chapter  offer  the 
best  means  of  evaluation. 
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The  subjective  part  of  this  approach  requires  each  staff 
section  chief  and  the  PM  to  estimate  the  value  of  each  factor, 
with  zero  being  the  least  important  and  a value  of  ten  being 
the  most  important.  The  analytical  part  involves  using  the 
sample  mean  of  these  estimates  as  the  value  of  relative  im- 
portance. Although  the  estimates  are  determined  subjectively, 
the  overall  approach  is  analytical  and  is  the  same  method  used 
by  the  communication  systems  described  in  the  last  chapter 
to  make  decisions.  Consequently,  an  accurate  assessment  of  the 
true  value  of  importance  is  obtained. 
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ESTIMATE  OP  THE  CIRCUMSTANCES 


In  today's  environment  the  complex  problems  of  program 
management  require  a logical  and  orderly  examination  of  all 
factors  affecting  the  resolution  of  a problem  to  reach  a sound 
decision.  How  a PM  arrives  at  a decision  is  a matter  for  his 
personal  determination.  However,  sound  decisions  result  best 

from  a thorough,  clear,  unemotional  analysis  of  all  pertinent 
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data.  Because  problems  associated  with  program  management  are 
complex,  the  organization  and  consideration  of  data  in  an 
orderly  sequence  make  a logical  analysis  easier  and  the  pro- 
duction of  sound  conclusions  more  likely.  This  orderly  sequence 
is  an  integral  part  of  what  is  called  the  Estimate  of  the  Cir- 
cumstances formulated  as  an  aid  for  the  PM  in  applying  thorough- 
ness, clarity,  judgment  and  professional  knowledge  to  each  sit- 
uation to  reach  a sound  decision. 

In  previous  chapters  management  and  analytical  techniques 
for  obtaining  and  evaluating  information  required  for  decision 
making  were  presented.  However,  these  techniques  in  themselves 
do  not  provide  the  necessary  integration  of  all  factors  that 
must  be  considered.  To  provide  the  integration  necessary  for 
obtaining  as  much  visibility  and  evaluation  of  all  factors 
related  to  making  a sound  decision  it  is  proposed  that  PMs 
adopt  the  use  of  a procedure  called  the  Estimate  of  the  Cir- 
cumstances. This  estimate  would  be  similar  to  the  estimate 
of  the  situation  procedure  now  being  used  by  combat  commanders 
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in  deciding  a course  of  action.  The  differences  between  the 
two  procedures  v/ould  be  in  names  of  the  paragraphs,  content 
and  the  method  of  analyzing  and  comparing  alternatives. 

The  estimate  of  the  circumstances  has  five  paragraphs. 

They  are  issue,  the  situation  and  alternatives,  analysis  of 
alternatives,  comparison  of  alternatives  and,  last,  the  de- 
cision paragraph.  This  last  paragraph  would  be  recommendations 
in  the  case  of  a staff  officer's  estimate  and  decision  in  the 
case  of  a PM's  estimate.  The  contents  of  each  paragraph  are 
discussed  in  succeeding  paragraphs. 

The  issue  paragraph  contains  a clear  and  concise  statement 
of  the  problem  or  in  other  words  what  must  be  decided. 

The  situation  and  alternatives  paragraph  contains  the 
overall  situation  and  feasible  alternatives  for  resolving  the 
issue.  There  are  two  purposes  fcr  considering  the  overall 
situation  in  paragraph  2 of  the  estimate:  the  first  is  to 
consider  all  elements  and  aspects  of  the  situation  that  affect 
the  issue  so  as  to  facilitate  development  and  analysis  of 
alternatives  and  the  second  is  to  formulate  feasible  alter- 
natives that  will  resolve  the  issue.  It  is  essential  to 
include  all  facts  and  assumptions  that  will  affect  the  issue. 
These  could  include  facts  pertaining  to  resources  available 
and  actual  or  potential  obstacles  to  success.  These  facts  are 
analyzed  separately  as  well  as  in  relation  to  one  another. 
Logical  deductions  are  made  as  to  the  effect  of  each  fact  on 
the  issue  and  on  the  determination  of  possible  alternatives. 

The  facts  obtained  in  paragraph  2 of  the  estimate  are  fur- 
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nished  to  the  PM  by  his  staff.  Paragraph  2 of  staff  estimates 
would  contain  facts  relative  to  their  respective  areas  of 
concern. 

The  purpose  of  the  third  paragraph,  analysis  of  alternatives, 
of  the  estimate  is  to  identify  the  advantages  and  disadvantages 
of  each  alternative,  to  add  refinements  and  embellishments, 
to  isolate  critical  factors  and  incidents  that  could  occur  during 
implementation  of  a decision  and  to  assist  ultimately  in  de- 
termining the  best  alternative. 

The  purpose  of  the  fourth  paragraph,  comparison  of  alter- 
natives, is  to  compare  the  alternatives  that  will  resolve  the 
issue,  i.e.,  those  alternatives  that  are  feasible,  so  that  a 
conclusion  as  to  the  best  alternative  can  be  reached.  In  this 
paragraph  the  information  contained  in  the  preceding  paragraphs 
is  analyzed  and  interpreted  into  a comprehensive  evaluation  of 
each  alternative.  The  facts,  assumptions,  cost  schedule,  and 
technical  data  and  general  circumstances  are  incorporated  into 
the  construction  of  a decision  tree.  The  advantages  and  dis- 
advantages are  evaluated  as  described  in  the  last  chapter. 

After  all  values,  submitted  by  the  staff  in  their  esti- 
mates, have  been  averared  and  added  to  determine  which  alter- 
native is  best  (highest  score)  and  the  decision  tree  analysis 
is  completed,  the  PM  has  adequate  information  to  make  a de- 
cision. The  results  of  these  two  methods  of  analysis  and 
evaluation  should  yield  the  same  solution  as  to  the  best  course 


of  action.  If  not,  recheck  all  information  and  calculations 
to  see  if  a mistake  occurred  in  the  tree  diagram  or  in  averaging 
and  addition  of  values.  If  no  mistake  is  found,  then  the  re- 
sults of  the  tree  diagram,  since  it  is  the  most  likely  and 
probably  the  best  outcome,  should  be  selected. 

The  last  paragraph  contains  the  decision  or  recommendation 
in  the  case  of  staff  estimates. 

If  used  properly  the  estimate  of  circumstances  provides 
a means  for  provoking  an  orderly,  logical,  thorough  and  com- 
prehensive thought  process  which  is  needed  to  resolve  the  many 
complex  problems  faced  by  a PM. 
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PROGRAM  MANAGEMENT  DECISIONS 


Program  Managers  are  charged  with  developing  and  producing 
a product  with  cost,  schedule  and  technical  performance  con- 
straints. Consequently,  decisions  made  by  a PM  must  take  into 
consideration  these  three  factors  to  insure  that  constraints 
are  not  exceeded.  So  far,  specific  problems  generally  faced 
by  a PM  have  not  been  discussed.  In  this  chapter  it  is  shown 
how  all  of  the  methods  and  procedures  previously  discussed 
can  be  related  some  of  the  critical  problems  that  occur  during 
the  life  cycle  of  a major  weapons  system.  Estimation  of  cost, 
schedules  and  technical  performance  criteria  will  be  discussed 
separately  and  it  is  shown  how  these  estimates  are  used  in  the 
proposed  Estimate  of  the  Circumstances.  Finally,  the  three 
most  critical  decisions,  should  development  proceed  to  valida- 
tion, full  scale  development  (PSD)  or  production,  will  be  ex- 
amined and  related  to  the  methods  and  procedures  advocated  in 
this  report. 

In  today's  environment  cost  has  become  a dominant  factor. 
Conrress  is  continually  increasing:  pressure  upon  DOD  and  the 
services  to  reduce  cost  and  cost  overruns  for  acquisition  of 
major  systems.  Recent  studies  by  the  GAO  indicate  that  the 
services  have  habitually  made  poor  decisions  throughout  life 
cycle  development.  The  results  of  these  poor  decisions  have 
in  some  cases  increased  cost  by  20C,s.  (7)  Therefore,  if  the  PM 
and  his  project  is  to  survive  cost  must  be  controlled. 
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To  control  cost  a PM  must  first  of  all  establish  realistic 
estimates  for  the  total  program  to  include  cost  of  each  develop- 
ment phase.  Once  these  thresholds  are  established  the  PHIS 
and  the  C/SCSC  procedures  provide  adequate  means  for  monitoring 
the  flow  of  funds.  Unfortunately,  neither  of  the  systems  can 
prevent  cost  overruns  or  cost  increases  due  to  unexpected  prob- 
lems that  occur.  Therefore,  a management  reserve  is  necessary 
to  cover  these  increases.  The  task  is  then  to  estimate  a 
program  cost  which  includ®  an  adequate  reserve. 

The  current  procedures  for  estimating  program  cost  through 
the  use  of  parametric, independent , independent  government  and  the 
PM's  cost  estimates  appear  to  be  adequate.  However,  to  establish 
the  estimate  to  be  used  in  the  development  concept  paper  (DCP)  it 
is  proposed  that  these  estimates  be  averaged  to  determine  the 
sample  mean  which  would  represent  the  expected  cost,  but  the 
uncertainties  could  cause  the  cost  to  vary.  Therefore,  the 
sample  variance  should  be  computed  to  determine  how  much  the 
cost  could  vary  and  it  is  proposed  that  two  standard  deviations 
be  added  to  the  expected  cost.  Recognizing  that  these  calcu- 
lations are  approximate  and  are  subject  to  errors,  the  percentage 
of  errors  can  be  computed.  If  the  error  is  5 or  10%  then  this 
percentage  of  the  exoected  value  should  be  added  to  the  sum  of 
the  expected  value  and  the  two  standard  deviations.  This  value 
then  becomes  the  estimated  cost.  This  procedure  does  not 
guarantee  that  the  cost  is  exact  but  it  does  enhance  the  possi- 
bility that  the  cost  would  not  exceed  this  value  particularly 
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since  95$  of  the  values  will  be  below  the  mean  plus  two  standard 
deviations  with  most  of  the  values  centered  around  the  mean.  (5) 
Since  the  values  are  expected  to  be  centered  at  the  mean  and 
with  the  two  standard  deviations  plus  the  correction  factor 
added  for  errors  an  adequate  management  reserve  should  be 
available  for  uncertainties. 

This  procedure  of  estimating  total  program  cost  should  be 
used  whenever  an  estimate  is  needed.  It  is  applicable  for  esti- 
mating cost  for  any  situation  such  as  what  should  an  ECP  cost. 

Comparing  this  procedure  to  design  of  communication  systems 
previously  mentioned  reveals  that  it  is  exactly  the  same.  Ex- 
pected values  plus  known  deviations  are  incorporated  in  estab- 
lishing decision  thresholds  used  in  computer  circuitry  which 
are  extremely  accurate. 

Schedules  (time)  for  completion  of  an  activity  can  be 
estimated  using  standard  formulas.  (2)  Using  these  formulas  and 
the  WBS,  the  network  system  wnich  contains  all  the  activities  to 
be  accomplished,  is  constructed.  In  addition  to  estimating  time, 
formulas  exist  for  predicting  the  probability  that  a schedule  can 
be  met.  The  same  procedures  outlined  for  estimating  cost  should 
be  used  to  establish  time  schedules  for  completion  of  an  activity 
or  the  entire  program  and,  as  mentioned  previously  in  reference  to 
establishing  a management  reserve  for  cost,  this  method  should 
also  create  a management  reserve  of  time. 
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Initial  technical  performance  parameters  are  not  estab- 
lished by  the  PI';  therefore,  he  is  not  required  to  estimate  or 
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predict  these  values.  But  he  is  required,  based  on  these  param- 
eters, to  determine  the  extent  of  testing  required  to  validate 
achievement  of  the  desired  degree  of  performance.  The  standard 
formulas,  referenced  previously,  can  be  used  to  provide  con- 

v 

siderable  insight  as  to  how  many  test  items  (prototypes)  and 
actual  tests  need  be  conducted  to  validate  performance.  Results 
of  tests  can  be  analyzed,  using  these  same  procedures,  to  deter- 
mine if  desired  performance  was  achieved  or,  in  the  case  of  a 
fly-off,  which  prototype  performed  best. 

In  some  cases  an  analysis  of  the  factors  just  described 
would  provide  adequate  information  for  making  a decision. 

i However,  for  complex  problems  the  Estimate  of  Circumstances 

i ' 

approach  should  be  used  for  integration  and  evaluation  of  all 
factors . 

Integration  of  the  cost,  schedule  and  technical  performance 
estimates  should  occur  in  paragraph  4.  An  explanation  of  how 
estimates  were  derived,  whether  they  are  advantages  or  dis- 
advantages and  their  relative  importance  should,  be  included. 
Although  these  factors  (cost,  etc.)  are  very  important  they 
should  be  thoroughly  analyzed  and  in  light  of  the  particular 
problem  to  be  resolved.  In  other  words , using  the  zero  to 
ten  scale  for  relative  importance,  these  factors  should  not 
automatically  receive  a score  of  ten  but  whatever  is  appropriate 
in  view  of  the  circumstances. 

Perhaps  the  most  critical  problems  or  decisions  a PM  must 
make  are  deciding  if  the  project  is  ready  to  proceed  to  vali- 
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dation,  to  PSD  or  production.  These  decisions  are  critical 


1 


and  proceeding  from  one  of  these  phases  to  the  other  could 
result  in  tremendous  overruns  which  habitually  occur  if  de- 
cisions are  premature.  (7) 

1 Many  factors  must  be  considered  before  proceeding  to  one 

of  these  phases.  However,  techniques  and  procedures  out- 
lined thus  far  are  ideal  for  analyzing  and  evaluating  complex 
problems.  A thorough  and  in  depth  Estimate  of  the  Circum- 
stances should  be  prepared  by  each  staff  section  and,  if 
properly  prepared  without  undue  emotion,  a logical  and  sound 
decision  is  likely  to  occur. 
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SUMMARY  AND  CONCLUSIONS 


Most  major  problems  encountered  by  a PM  are  directly 
related  to  cost,  schedule,  and  technical  performance  and  be- 
cause these  factors  are  interdependent,  an  optium  sound  de- 
cision is  difficult  to  make.  However  difficult  and  complex, 
decisions  must  be  made  and  often  must  be  made  within  a re- 
latively short  time  period.  Therefore,  a PM  must  have  a sys- 
tem that  provides  an  orderly  and  logical  evaluation  of  all 
factors  related  to  a problem.  Such  a system  increases  the 
possibility  of  a PM  making  the  best  possible  decision. 

The  methods  and  procedures  advocated  in  this  report 
provide  a system  that  effectively  considers  and  integrates 
all  factors  and  thereby  produces  adequate  information  for 
a PM  to  make  a decision.  First,  the  PMIS  and  C/SCSC  predicts 
cost  and  schedule  data  related  to  development  of  tne  hard- 
ware. Next,  the  network  system  provides  an  overall  view 
(schedule)  of  all  activities  associated  with  the  project. 

With  this  system  interrelationship  of  activities  and  subse- 
quent events  or  changes  to  schedules  can  be  accessed.  It 
was  also  shown  that  analytical  techniques  used  for  sampling, 
estimation  and  prediction  are  powerful  techniques  and  should 
be  used  wherever  possible.  Last,  the  Estimate  of  the  Circum- 
stances was  shown  to  be  an  effective  means  for  integrating  all 
factors . 

These  procedures  are  directly  analogous  to  procedures  used 
to  design  complex  and  extremely  accurate  electronic  systems 


particularly  computers.  Therefore,  it  can  be  reasonably  con- 
cluded that  since  many  uncertainties  are  accurately  judged  and 
predicted  in  these  devices,  using  these  techniques,  uncer- 
tainties faced  by  a PM  can  be  handled  in  the  same  manner.  Con- 
sequently, the  proposed  Estimate  of  the  Circumstances  along  with 
the  analytical  methods  greatly  enhance  the  ability  of  a PM  to 
maKe  sound  logical  decisions  during  all  phases  of  life  cycle 
development  of  a major  weapons  system. 
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